Skip to content

fix(release): guard deferred fallback preparation and parallelize portfolio qualification - #9168

Merged
loopx-agent merged 1 commit into
mainfrom
codex/fix-240-qual-and-speed-20261121
Nov 21, 2026
Merged

loopx-agent merged 1 commit into
mainfrom
codex/fix-240-qual-and-speed-20261121

Conversation

@loopx-agent

@loopx-agent loopx-agent commented Nov 21, 2026 •

Copy link
Copy Markdown
Collaborator

Summary

Fixes the remaining v2.4.0 exact-source qualification regressions surfaced on main and adds opt-in --max-workers concurrency to live Doubao portfolio qualification:

  1. Deferred successor replan execution-preparation guard (loopx/control_plane/quota/execution_preparation.py): Exclude effective_action == "successor_replan_required" and fallback_kind == "deferred_successor_replan" from scoped_fallback_admitted in apply_selected_todo_execution_preparation, preventing execution_preparation.ts from rejecting ready deferred lifecycle-replan fallbacks with selected_todo_not_open while preserving hard-lease preparation for open scoped_user_gate_fallback deliveries.
  2. Deep-path host tool error recovery capture (loopx/control_plane/testing/replan_semantic_action_behavior.py): Raise _MAX_HOST_TOOL_ERROR_OUTPUT_CHARS from 4096 to 8192 so replan_writeback_rejection_v0 JSON payloads (exceeding 4 KB under deep pytest tmp_paths) are not truncated mid-JSON before context_recovery.select_command.
  3. Architecture & smoke alignment (tests/architecture/test_content_digest_single_owner.py, examples/bootstrap-command-pack-smoke.py, tests/control_plane/test_effect_turn_live_quota_decision.py): Pin loopx.cli_commands.quota_capture and loopx.extensions.lark.private_delivery_discovery (removing retired loopx.extensions.lark.private_conversations), align multi-goal host_loop_activation smoke expectation with PR fix(release): stabilize CLI help, architecture guards, onboarding gate, and public smokes #9152 (host_surface: null, activation_method: null), and package-qualify tests.control_plane.canonical_authority_fixture so tests/test_host_launch_authority.py collects standalone.
  4. Opt-in parallel live portfolio execution (loopx/control_plane/testing/actual_default_model_behavior_portfolio.py, scripts/qualify-doubao-model-behavior-live.py): Add max_workers: int = 1 and --max-workers (default 1 preserves byte-identical sequential behavior; >1 runs independent scenarios across a bounded ThreadPoolExecutor with lock-protected abort/counters while preserving _SCENARIOS catalog receipt order).
  5. Release notes reconciliation (docs/product/releases/v2.4.0.md): Reconcile EN/ZH notes with #9167 and #9168.

Verification (run in clean worktree)

  • pytest -q -n 8 across all 9 affected/adjacent test modules (test_host_launch_authority.py, test_effect_turn_live_quota_decision.py, test_scoped_gate_successor_tool_behavior.py, test_actual_default_model_behavior_portfolio.py, test_quota_execution_preparation.py, test_replan_semantic_action_behavior.py, test_content_digest_single_owner.py, test_scoped_gate_fallback_cli.py, test_settled_monitor_user_gate.py): 591 passed, 0 failed.
  • Isolated examples/bootstrap-command-pack-smoke.py: passed.
  • examples/release/release-readiness-doc-smoke.py --release-notes docs/product/releases/v2.4.0.md across all 12 surfaces: passed.
  • ruff check, mypy, scripts/check_rfc_bilingual.py --base origin/main --staged, and loopx check across all 11 changed files: all passed (0 errors, 0 warnings).

@loopx-agent
loopx-agent force-pushed the codex/fix-240-qual-and-speed-20261121 branch from cc7808c to 1df7be9 Compare November 21, 2026 05:45

@loopx-agent loopx-agent left a comment

Copy link
Copy Markdown
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Approval conclusion (author-owned PR; GitHub blocks formal self-approval)

Reviewer: model_agent · Gemini · Google (self_reported)

动机 / Motivation

受影响调用方与触发场景: 启用 hard_lease 或运行 v2.4.0 发布验收与真实模型行为评估的维护者和自动化调度流程。

改前与改后对比: 此前在作用域 user_gate 旁路选中就绪的延迟后继待办时,执行准备守卫误将其当成待执行任务送入租约投影,导致未开放状态被拒绝并清零运行资格;深路径下重规划写回拒收 JSON 超过 4096 字符而被截断;同时架构摘要清单、多 Goal 引导测试及独立测试导入与 main 脱节,实模组合只能串行执行。修复后,延迟后继重规划跳过租约准备并保留原有 open 回退的 hard_lease 校验,错误输出上限增至 8192 字符,测试与发布说明对齐,并新增默认关闭的 --max-workers 并发选项。

可观测收益: 在受保护的 SQLite 真实路径与单元测试中,deferred_successor_replan 保持 should_run=true 且不触发 selected_todo_not_open,open 作用域回退仍严格要求 claim_required;深路径 writeback rejection 恢复测试通过;run_actual_default_model_behavior_portfolio 在 max_workers=1 时保持顺序与回执完全一致,在 max_workers>1 时按目录顺序输出等价结果;v2.4.0.md 与 #9167/#9168 中英文发布说明同步。

本 PR 边界(Non-goals): 本 PR 不改动 execution_preparation.ts 的状态机规则,不改变默认 max_workers=1 顺序执行行为,也不在无凭证或隔离工作树中自动运行在线 Ark 接口。

若放任主干现状,v2.4.0 在启用 hard_lease 的真实施工环境中会把受 user_gate 旁路选出的就绪延迟后继误判阻断,深路径环境下的重规划拒收恢复链路会持续丢包截断,独立单文件测试收集与多 Goal 引导回归也会继续报错,同时 22 场景重复 2 次的真实模型组合评估耗时过长。此切片范围完整、可直接回归验证且低风险可逆。

改动思路 / Approach

本次改动遵循最小修复与现有职责归属原则(exact head 1df7be9328abcdc4ae4b0335b2dd718547a271a9):

  1. 延迟后继重规划与执行租约边界分离:在 apply_selected_todo_execution_preparation 入口,当 effective_action == "successor_replan_required"(复用 AgentScopeFrontierAction.SUCCESSOR_REPLAN_REQUIRED)或 scoped_user_gate_fallback.selected_executable.fallback_kind == "deferred_successor_replan" 时,不再将其视为 scoped_fallback_admitted,从而不向 TypeScript projectExecutionPreparation 发起针对 status="deferred" 待办的租约查询。原有的 open 作用域回退仍照常进入 hard_lease 准备并强制 claim_required。
  2. 宿主工具错误输出截断上限提升:将 replan_semantic_action_behavior.py 中 _HostToolError 的输出截断常量由魔法数字 4096 抽取并提升为 _MAX_HOST_TOOL_ERROR_OUTPUT_CHARS = 8192,保证较深临时目录路径下的 replan_writeback_rejection_v0 完整 JSON(含 context_recovery.select_command)不被腰斩。
  3. Opt-in 并发组合执行:run_actual_default_model_behavior_portfolio 与 scripts/qualify-doubao-model-behavior-live.py 新增默认值为 1 的 max_workers / --max-workers。max_workers == 1 走原顺序分支;max_workers > 1 使用有界 ThreadPoolExecutor 与 threading.Lock 保护 aborted 状态和全局计数集合,并借 pool.map 保留 _SCENARIOS 的确定性输出顺序。

四项运行时与测试改动各自停留在既有职责边界内:作用域延迟重规划过滤留在 Python 适配层 apply_selected_todo_execution_preparation,错误输出截断上限留在 replan_semantic_action_behavior 的宿主循环,并发执行留在真实模型评估组合及 live 脚本入口,不引入新的状态存储、协议字段或平行决策层。 当前 PR 仅修复已暴露的发布验收回归并增加 opt-in 并发参数,保留 TypeScript execution_preparation 对 open Todo 的既有语义、sequential 默认值(max_workers=1)与现有词汇表。

场景 / 路径 改前行为(origin/main) 改后行为(1df7be9328ab) 保护边界
scoped_user_gate_fallback + deferred_successor_replan 误判 scoped_fallback_admitted=True,被 TS 拒为 selected_todo_not_open(should_run=False) 排除租约准备,保持 should_run=True 与 successor_replan_required open 作用域回退仍走 hard_lease 准备
深路径 _HostToolError 返回 exc.output[:4096] 截断 replan_writeback_rejection_v0 JSON 提升至 8192 字符(_MAX_HOST_TOOL_ERROR_OUTPUT_CHARS) 非错误正常响应路径不变
真实模型组合评估执行 仅串行执行(22 * 2 次调用) 默认 max_workers=1 保持全等;>1 有界线程池并发且保持顺序 禁止 bool 及 <1 参数

具体改动 / Changes

  • 规范依据说明(spec_basis):本 PR 针对发布前发现的回归缺陷与测试工具并发优化,无独立新增 RFC(no_spec);其中 docs/product/releases/v2.4.0.md 对照 docs/product/release-readiness.md 的双语发布说明契约校验通过,execution_preparation.py 对照既有 execution_preparation.ts 与 agent_scope.py 状态契约。

关键代码讲解

  1. apply_selected_todo_execution_preparation(loopx/control_plane/quota/execution_preparation.py:448-475)

    • 职责:在 should_run=True 时为正常交付或开放态 scoped_user_gate_fallback 注入 execution_preparation(hard_lease 就绪/认领门禁)。
    • 修复逻辑:读取 fallback = payload.get("scoped_user_gate_fallback") 及其 selected_executable,在 scoped_fallback_admitted 判定中新增两个收敛条件:payload.get("effective_action") != AgentScopeFrontierAction.SUCCESSOR_REPLAN_REQUIRED.value 与 fallback_selected.get("fallback_kind") != "deferred_successor_replan"。两者中任意一项成立即退出租约准备,避免把仅需生命周期重规划(reopen / supersede / no-follow-up)的 deferred 待办送入只接受 open 待办的 TypeScript projectExecutionPreparation。
    • 保留边界:当选中项是普通 open 作用域回退(fallback_kind 非 deferred_successor_replan 且 effective_action 非 successor_replan_required)时,依然触发 project_selected_todo_execution_preparation,并在缺少活跃租约时返回 claim_required。
  2. run_actual_default_model_behavior_portfolio(loopx/control_plane/testing/actual_default_model_behavior_portfolio.py:1393-1509)

    • 职责:编排 22 个真实模型行为场景(每场景 2 轮重复)及 6 项不变性/敏感性对比(contrasts)。
    • 改动细节:先对 max_workers 做严格类型校验(拒绝 bool、非 int 与 < 1);抽出 _aborted_scenario_result 与 _run_one 消除重复逻辑。max_workers == 1 完全走原有串行逻辑;max_workers > 1 时创建上限为 min(max_workers, len(_SCENARIOS)) 的 ThreadPoolExecutor,在 _worker 内通过 threading.Lock 读写 aborted,利用 pool.map(_worker, _SCENARIOS) 保证返回的 results 与 _SCENARIOS 顺序一致,并在收集循环中仅对非 "not_run" 的场景写入 observations 和累加 actor_call_count。
  3. _run_qualification_loop(loopx/control_plane/testing/replan_semantic_action_behavior.py:57, 1760-1766)

    • 职责:驱动带真实 CLI / 沙箱 Shell 的模型行为验收循环。
    • 改动细节:引入常量 _MAX_HOST_TOOL_ERROR_OUTPUT_CHARS = 8192,替换原来硬编码的 4096,避免在较深 tmp_path(如多级目录叠加 64 字符后缀)下 refresh-state 返回的 replan_writeback_rejection_v0 载荷在 context_recovery.select_command 字段前被截断。
  4. main 与 counted_transport(scripts/qualify-doubao-model-behavior-live.py:60-173)

    • 职责:真实 Ark Doubao 模型行为评估的 CLI 入口与请求统计包装。
    • 改动细节:新增 --max-workers(默认 1,< 1 时抛 ValueError),并用 transport_lock = threading.Lock() 保护共享的 provider_call_count 与 provider_models 集合。各场景和重复轮次通过 sha256(run_id)[:16] 隔离临时夹具目录,避免并发写盘冲突。

其他对齐改动

  • 摘要消费者清单与导入路径:tests/architecture/test_content_digest_single_owner.py 将新增使用 BARE_SHA256_PATTERN 的 loopx.cli_commands.quota_capture 和 loopx.extensions.lark.private_delivery_discovery 纳入 CONSUMER_MODULES 并移除已迁出的 loopx.extensions.lark.private_conversations;tests/control_plane/test_effect_turn_live_quota_decision.py 将 canonical_authority_fixture 改为完整包路径 tests.control_plane.canonical_authority_fixture,恢复 tests/test_host_launch_authority.py 独立收集与执行。
  • Smoke 与测试覆盖:examples/bootstrap-command-pack-smoke.py 对齐 PR #9152 在多 Goal goal_selection_required 分支中新增的 "host_surface": None, "activation_method": None;test_quota_execution_preparation.py、test_actual_default_model_behavior_portfolio.py、test_replan_semantic_action_behavior.py 均新增对应正反例与回归测试。
  • 发布说明双语同步:docs/product/releases/v2.4.0.md 同步补充 #9167(回环中继 401 恢复)与 #9168(延迟后继排除租约准备、--max-workers、8 KiB 错误输出上限、摘要清单与引导 smoke/导入对齐)的英文与中文条目。

对主干的风险 / Risks

  1. Opt-in 并发隔离与反事实校验(default_off_isolation & scope_coverage):
    • max_workers=1(默认值)保持既有顺序执行流,不启用 ThreadPoolExecutor,输出回执字节级等价;max_workers=4 在测试与故障注入反事实(一个场景抛出 RuntimeError)下均保持结果顺序确定性、not_run 跳过统计正确且无锁竞争或崩溃。需注意:在 max_workers > 1 并发模式下若某一场景发生 actor_error,已在线程池中开始执行的同批在途场景会跑完自身当前轮次,之后尚未启动的场景才会观测到 aborted=True 并标记为 not_run(这是有界并发的合理语义,对串行模式无影响)。
    • 在真实 SQLite hard_lease 隔离夹具上执行了成对反事实校验:当作用域 user_gate 选中 deferred_successor_replan 时,quota should-run 保持 should_run=True、effective_action="successor_replan_required" 且不生成 execution_preparation;当同一 Todo 通过 todo update --status open 变为 open 回退时,仍准确触发 execution_preparation.state == "claim_required"(task_lease_missing),未破坏开放任务的硬租约门禁。
  2. 线程安全边界检查:
    • 核查了 7 个行为 actor(DoubaoModelBehaviorActor、DoubaoOnboardingModelBehaviorActor、DoubaoSelectedTodoToolBehaviorActor、DoubaoReplanSemanticActionBehaviorActor、DoubaoScopedGateSuccessorToolBehaviorActor、DoubaoCapabilityMonitorRepairToolBehaviorActor、DoubaoTerminalSettlementToolBehaviorActor)、VisionShellHost、execute_loopx_cli 与 effect_runtime.py:actor 实例无可变共享状态,每个 run_id 使用独立临时目录、AF_UNIX 套接字和子进程环境,TS Effect Runtime 通过指纹文件锁安全复用。
  3. 语义对齐与本地验证矩阵:
    • 词汇表复用(reuse_existing):未新增任何枚举、状态字面量或配置键,直接复用 AgentScopeFrontierAction.SUCCESSOR_REPLAN_REQUIRED 与既有的 fallback_kind == "deferred_successor_replan"。
    • 隔离工作树内 examples/semantic-vocabulary-drift-smoke.py 因未安装 node_modules(typescript 开发依赖缺失)而退出失败(NPM_DEV_DEPENDENCIES_MISSING);鉴于本 PR 未改动任何 .ts 文件或语义词汇定义文件,该环境限制不阻塞评审结论。其余已执行验证全部 PASS(见下表):
验证项 状态 观察结果
10 个核心及关联 pytest 模块(pytest -q -n 8) PASS 605 passed in 142.17s(含新增 3 个回归测试及 test_host_launch_authority.py 独立收集)
3 项新增回归用例定向复跑(pytest -v -k ...) PASS deferred_replan 排除、max_workers=4 等价性、深路径恢复均通过
TypeScript 单测(execution_preparation, interaction_contract, scoped_gate_fallback) PASS 44/44 passed(node --experimental-strip-types --test)
examples/bootstrap-command-pack-smoke.py(隔离 HOME) PASS bootstrap command pack smoke passed
examples/release/release-readiness-doc-smoke.py(12 个可选能力面) PASS release-readiness-doc-smoke ok
ruff check(全部变更 Python 文件)& mypy(配置严格基线 17 文件) PASS 0 errors
scripts/check_rfc_bilingual.py & loopx check --scan-path(11 文件) PASS 双语 RFC 检查与公开边界扫描 0 errors
loopx canary premerge --from-git-diff --git-diff-base origin/main FAIL(环境依赖) 18/19 通过;仅 semantic-vocabulary-drift-smoke.py 因无 node_modules 报错
真实 SQLite hard_lease 正/反对比 & max_workers=4 异常中止反事实 PASS deferred 跳过租约、open 仍需 claim_required、并发中止无异常
  1. 轻微非阻塞说明:docs/product/releases/v2.4.0.md 中将 8192 字符上限写为 8 KiB(在纯 ASCII JSON 载荷下数值等价,对多字节非 ASCII 字符略偏口语化);_bound_semantic_writeback 以外的 terminal_settlement_tool_behavior.py 仅对 workspace_shell 保留 4096 截断且不走 refresh-state JSON 输出,不影响恢复链路。

我的整体评价 / Verdict

  • 长程与体验影响(outcome_impact):long_horizon 判定为 preserved(修复了作用域网关旁路下延迟后继被错误阻断和深路径重规划恢复被截断的回归,同时保留开放任务的 hard_lease 防线);user_experience 判定为 improved(提供 opt-in --max-workers 并发加速评估并修复独立测试收集及引导 smoke 漂移)。本 PR 仅涉及控制面 Python 逻辑、测试工具与发布文档,未修改任何渲染界面,因此 visual_review 判定为 not_applicable(checked scope: full diff across 11 files, unverified_scope: none)。
  • 代码体量、架构适配与语义结论:变更规模克制(+220/-42 across 11 files),无多余兼容分支或重复状态,架构归属评估(architecture_assessment)为 retain——四项运行时与测试改动各自停留在既有职责边界内:作用域延迟重规划过滤留在 Python 适配层 apply_selected_todo_execution_preparation,错误输出截断上限留在 replan_semantic_action_behavior 的宿主循环,并发执行留在真实模型评估组合及 live 脚本入口,不引入新的状态存储、协议字段或平行决策层。 当前 PR 仅修复已暴露的发布验收回归并增加 opt-in 并发参数,保留 TypeScript execution_preparation 对 open Todo 的既有语义、sequential 默认值(max_workers=1)与现有词汇表。 语义对齐上,PR 完全 reuse_existing(复用既有 AgentScopeFrontierAction.SUCCESSOR_REPLAN_REQUIRED 与 deferred_successor_replan,无新词汇引入)。
  • 残留风险:实网 Ark API 在 --max-workers > 1 下的限流表现未在本次本地无密钥隔离审查中实测;隔离工作树缺少 node_modules 导致 semantic-vocabulary-drift-smoke.py 的 TypeScript AST 扫描未在本地跑通(人工审查确认无 .ts/词汇变更)。
  • 精确 Head 确认:已于发布前核验精确提交 1df7be9328abcdc4ae4b0335b2dd718547a271a9,本地核心验证完备且未发现阻塞主干合并的缺陷。

English verdict: APPROVE at exact head 1df7be9. All 11 changed files cleanly fix the scoped deferred-replan execution-preparation regression, deep-path host-tool rejection truncation, digest manifest/test-import/smoke drift, and v2.4.0 release-note reconciliation while keeping --max-workers=1 default-off isolation and open-fallback hard-lease guarantees intact. Verified with 605 passed pytest cases, 44 passed TS tests, isolated bootstrap and 12-surface release-readiness smokes, ruff/mypy/public-boundary checks, and paired SQLite hard-lease counterfactuals (premerge canary semantic-vocabulary-drift-smoke.py failed only due to absent node_modules in the isolated worktree; no .ts or vocabulary changes in PR).

@loopx-agent
loopx-agent merged commit d115bd7 into main Nov 21, 2026
21 of 46 checks passed
@loopx-agent
loopx-agent deleted the codex/fix-240-qual-and-speed-20261121 branch November 21, 2026 06:04
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant